MLOps: Automating the Machine Learning Stack

Myles Mitchell @ Jumping Rivers

Welcome!

Virtual environment

https://vetiver-mlops.jumpingrivers.training/welcome/

Password: passionfruit-pineapple

Screenshot of log in page

Before we start…

Who am I?

  • Background in Astrophysics and Data Science.

  • Principal Data Scientist @ Jumping Rivers:

    • Project management.

    • Python and ML support for clients.

    • Maintains and teaches courses in Python, SQL, ML.

  • Hobbies include hiking and travelling.

Jumping Rivers

↗ jumpingrivers.com   𝕏 @jumping_uk

  • Machine learning
  • Dashboard development
  • R packages and APIs
  • Data pipelines
  • Code review
     

How will this workshop run?

  • Presentation slides (bit.ly/2025-rss-mlops).

  • The slides use R code.

  • A Python version of the live demo is also available.

  • The exercises will apply the same techniques to a different data set and model.

  • What type of Workbench Session / IDE to use?

    • R users: RStudio
    • Python users: Jupyter

What will be covered?

  • Introduction to MLOps

  • Building a basic MLOps workflow

  • We will be deploying models locally only…

    • Means all tools are open source and free!

  • Want to contribute?

    • Raise your hand for questions and comments at any time!

What you get from me

Introduction to MLOps

Let’s take a step back…

The typical data science workflow:

Typical data science workflow. Starting with data importing and tidying, followed by a cycle of data transformation, data visualisation and modelling which repeats as the model is better understood. The results from this cycle are then communicated.
  • Data is imported and tidied.
  • Cycle of data transformation, data visualisation and modelling.
  • The cycle repeats as we understand the underlying model better.
  • The results are communicated to an external audience.

From Classical Stats to Machine Learning

  • The classical workflow prioritises understanding the system behind the data.
  • By contrast, Machine Learning prioritises prediction.
  • As data grows we reconsider and update our ML models to optimise predictive power.
  • A goal of MLOps is to streamline this cycle.

What is MLOps?

MLOps: Machine Learning Operations

MLOps workflow. Starting with data importing and tidying, followed by modelling and finishing with model versioning, deployment and monitoring. The cycle then repeats as more data is acquired.
  • Framework to continuously build, deploy and maintain ML models.
  • Encapsulates the “full stack” from data acquisition to model deployment.
  • Monitor models in production and detect “model drift”.
  • Versioning of models and data.

MLOps frameworks

  • Amazon SageMaker
  • Google Cloud Platform
  • Kubeflow (ML toolkit for Kubernetes)
  • Vetiver by Posit (free to install, nice for beginners)
  • And the list goes on…

     

Vetiver

  • Open source tool maintained by Posit (formerly RStudio).
  • Integrates with popular ML libraries in R and Python.
  • Fluent tooling to version, deploy and monitor a trained model.
  • Supports deploying models to localhost - great way to learn MLOps!

Your first MLOps pipeline

Let’s build an MLOps stack!

  • Data
  • Modelling
  • Deployment
  • Monitoring
  • Repeat

Modelling penguins

Let’s set up a basic MLOps workflow!

  • Palmer Penguins dataset:

    library("palmerpenguins")

names(penguins)
    [1] "species"           "island"            "bill_length_mm"   
[4] "bill_depth_mm"     "flipper_length_mm" "body_mass_g"      
[7] "sex"               "year"
Scatter plot showing positive relationship between penguin flipper length and penguin body mass. The data points are coloured based on species and shaped based on island. The Gentoo penguins tend to have higher body mass and flipper length than Adelie and Chinstrap.

Palmer Penguin dataset

Data tidying and cleaning

Let’s predict species using flipper length, body mass and island!

  • Using {tidyr} and {rsample}:

    # Drop missing data
penguins_data = tidyr::drop_na(penguins)

# Split into train and test sets
penguins_split = rsample::initial_split(
  penguins_data, prop = 0.8
)
train_data = rsample::training(penguins_split)
test_data = rsample::testing(penguins_split)

Demo

Open demo.R in RStudio / demo.ipynb in Jupyter…

Task 1: Data loading and tidying

  • Open exercises.R / exercises.ipynb

  • Attempt “Task 1: Data loading and tidying”

  • Need help? Check solutions.R / solutions.ipynb or raise your hand

08:00

Data best practices

  • Move from large CSVs to more efficient formats like Parquet and Arrow.
  • Tools like Apache Spark can speed up data processing.
  • Add a data validation step.
  • Version your data.
  • Your preferred ML platform probably has built-in tools for data wrangling.

Modelling

  • Let’s set up the model recipe in {tidymodels}:
library("tidymodels")

model = recipe(
  species ~ island + flipper_length_mm + body_mass_g,
  data = train_data
) |>
  workflow(nearest_neighbor(mode = "classification")) |>
  fit(train_data)

Inference

Our model object can now be used to predict species:

model_pred = predict(model, test_data)

# Accuracy for unseen test data
mean(
  model_pred$.pred_class == as.character(
    test_data$species
  )
)
[1] 0.8358209

Enter Vetiver!

  • Use a Vetiver model object to collate all of the info needed to store, deploy and version our model:

    v_model = vetiver::vetiver_model(
  model,
  model_name = "k-nn",
  description = "penguin-species"
)
v_model
    
── k-nn ─ <bundled_workflow> model for deployment 
penguin-species using 3 features

Vetiver model

v_model is a list with six elements

  • View the contents:

    names(v_model)
    [1] "model"       "model_name"  "description" "metadata"    "prototype"  
[6] "versioned"

  • View the model description:

    v_model$description
    [1] "penguin-species"

  • View the metadata:

    v_model$metadata
    $user
list()

$version
NULL

$url
NULL

$required_pkgs
[1] "kknn"      "parsnip"   "recipes"   "workflows"

Demo

Go back to demo.R in RStudio / demo.ipynb in Jupyter…

Task 2: Modelling

  • Open exercises.R / exercises.ipynb

  • Attempt “Task 2: Modelling”

  • Need help? Check solutions.R / solutions.ipynb or raise your hand

10:00

Modelling best practices

  • Consider packaging source code to encourage proper documentation, testing and dependency management.
  • Consider auto-ML tools like H2O.ai and SageMaker Autopilot for model selection.
  • Version and store your models for reuse later.

Model versioning

  • Use {pins} to store R or Python objects for reuse later.

  • Store pins using “boards” including Posit Connect, Amazon S3 or even Google drive!

  • Storing in a temporary directory:

    model_board = pins::board_temp(versioned = TRUE)
model_board |> vetiver::vetiver_pin_write(v_model)

Retrieving a pinned model

  • Retrieve a model

    model_board |> vetiver::vetiver_pin_read("k-nn")
    
── k-nn ─ <bundled_workflow> model for deployment 
penguin-species using 3 features

  • Inspect the stored versions

    model_board |> pins::pin_versions("k-nn")
    # A tibble: 1 × 3
  version                created             hash 
  <chr>                  <dttm>              <chr>
1 20250901T152042Z-7e0b6 2025-09-01 16:20:42 7e0b6

Model deployment

Deployment using Vetiver

  • We deploy models as APIs which take input data and send back model predictions.

  • We can use a {plumber} API (R) or FastAPI (Python) to deploy a {vetiver} model.

Deploying locally

  • {vetiver} and {plumber} support local deployment:

    plumber::pr() |>
  vetiver::vetiver_api(v_model) |>
  plumber::pr_run()

  • Opens the API in a browser window

  • Great for beginners to MLOps and APIs!

Deploying locally

Check the deployment with:

base_url = "127.0.0.1:8080/"  # update the port number!
url = paste0(base_url, "ping")
r = httr::GET(url)
metadata = httr::content(
  r, as = "text", encoding = "UTF-8"
)
jsonlite::fromJSON(metadata)

Model predictions

Checking that our API works!

  • Endpoints metadata and predict allow programmatic queries:

    url = paste0(base_url, "predict")
endpoint = vetiver::vetiver_endpoint(url)
pred_data = test_data |>
  dplyr::select(
    "island", "flipper_length_mm", "body_mass_g"
  ) |>
  dplyr::slice_sample(n = 10)
predict(endpoint, pred_data)

Demo

Go back to demo.R in RStudio / demo.ipynb in Jupyter…

Task 3: Deploying your model

  • Open exercises.R / exercises.ipynb

  • Attempt “Task 3: Deploying your model”

  • Need help? Check solutions.R / solutions.ipynb or raise your hand

05:00

Aside: What about Python?

  • Vetiver is available for both Python and R!

  • In Python you would use Python ML libraries rather than {tidymodels}

    • scikit learn
    • PyTorch
    • XGBoost
    • statsmodels

  • Vetiver documentation: vetiver.posit.co

Deployment best practices

  • Try deploying locally to check that your model API works as expected.

  • Use environment managers like {renv} to store model dependencies.

  • Use containers like Docker to bundle model source code with dependencies.

Deploying to the cloud

  • Vetiver also streamlines deployment to the production environment:

    vetiver::vetiver_prepare_docker(
  pins::board_connect(), 
  "myles/k-nn", 
  docker_args = list(port = 8080)
)

  • This command:

    • Lists R depedencies with {renv}

    • Stores the {plumber} API code in plumber.R

    • Generates a Dockerfile

Dockerfiles

Our Dockerfile contains a series of commands to:

  • Set the R version and install the system libraries.

  • Install the required R packages.

  • Run the API.

Running Docker

  • Build a Docker container:

    docker build --tag my-first-model .

  • Inspect your stored Docker images:

    docker image list

  • Run the image:

    docker run --rm --publish 8080:8080 my-first-model

  • These steps can be run in sequence using a CI/CD pipeline.

Deploying to Connect

  • Vetiver integrates nicely with Posit Connect:

    vetiver::vetiver_deploy_rsconnect(
  board = pins::board_connect(), "myles/k-nn"
)

  • We can also publish to Amazon SageMaker using vetiver_deploy_sagemaker()

  • For other cloud platforms check the process for pushing Docker contains

Cost considerations for cloud MLOps

  • Some platforms offer free trials (e.g. SageMaker).

  • May be cheaper if you’re already invested in a particular cloud platform.

  • Costs can rise depending on computational resources consumed.

  • Model building and deployment use different environments.

Monitoring your model

Deployment is just the beginning…

Model drift

  • Model performance may drift as the data evolves…
    • Data drift: statistical distribution of input feature changes.
    • Concept drift: relationship between target and input variables changes.
  • The context in which your model was trained matters!

Task 4: Detecting model drift

  • Open exercises.R / exercises.ipynb

  • Attempt “Task 4: Detecting model drift”

  • Need help? Check solutions.R / solutions.ipynb or raise your hand

08:00

Model monitoring

  • As your data grows, run regular checks of model performance.

  • Monitor key model metrics over time.

  • You may notice a downward trend…

  • Retrain the model with the latest data and redeploy.

Monitoring demand

As data and user base grows, your model needs to scale.

  • Upgrade your computational resources.

  • Consider moving from a relational database to a data warehouse.

  • Check how many users your license (AWS, Posit, etc) permits.

Monitoring with Vetiver

Vetiver has built in functions to track scoring metrics over time.

  • Requires a time variable in the dataset.

  • Load the model from your {pins} board.

  • Make sure you are scoring the deployed version.

  • Specify the period for scoring (weeks, months, years, …).

  • Model metrics can also be stored with {pins}!

Monitoring with Vetiver

Consider our life expectancy data from the exercises…

  • Compute scoring metrics over specified period:

    new_metrics = vetiver::augment(
  v_model, new_data = recent_data
) |>
  vetiver::vetiver_compute_metrics(
    Date, "year", `Life expectancy`, .pred
  )

  • Requires a Date column (generate from Year).

  • recent_data: could be data from the past year or all historical data.

Monitoring with Vetiver

  • Pin the metrics

    model_board |>
  vetiver::vetiver_pin_metrics(
    new_metrics, "k-nn_metrics", overwrite = TRUE
  )

  • Plot the metrics

    library("ggplot2")

monitoring_metrics = model_board |>
  pins::pin_read("k-nn_metrics")
vetiver::vetiver_plot_metrics(monitoring_metrics) +
  scale_size(range = c(2, 4))

Closing thoughts

Benefits of an MLOps Workflow

  • Retraining and redeployment can happen at the click of a button.

  • Encourages good practices like model versioning and packaging of source code.

  • Reduces human error.

  • Well defined and reproducible.

  • Consider whether it is worth the cost/effort before starting.

Thanks for listening!

Any questions?

We’re here all week!

  • Visit us at the Jumping Rivers stand

  • Check out my Thursday workshop: “Dynamic Presentations with Quarto”

    • Like Vetiver, Quarto is also maintained by Posit and multilingual by design

    • Render HTML, PDF and Word documents that combine plain text and code

    • Create your first slide deck in Quarto!

  • Check out Aida’s lightning talk!

Shiny In Production 2025

  • Join us for Shiny In Production (8-9 Oct), Newcastle Upon Tyne

  • shiny-in-production.jumpingrivers.com

  • A half day of workshops followed by a day of talks on all things Shiny

  • Discount code (30% off): RSS2025

Free monthly webinars

  • Jumping Rivers is organising free, monthly webinars!

  • Upcoming topics include Scalable Shiny Apps, Machine Learning with Python, Introduction to the Posit Ecosystem

  • Next webinar: 18 September

  • Find out more at jumpingrivers.com/blog/jumping-rivers-webinar-launch/

  • Follow “Jumping Rivers Ltd” on LinkedIn to keep up to date